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A  three  factorial  6M-  cell  fixed  effects  experimental  design  was  created  to 
investigate  the  impact  of  four  different  follow-up  methods »  two  different 
lengths  of  the  questionnaire  and  eight  distinct  markets  on  the  response  rate. 
The  telephone  reminder  was  significantly  better  and  the  telephone  interview 
without  alert  was  significantly  worse  in  generating  returns.  The  questionne 
length  had  no  effect  on  the  response  rate*  Finally ,  the  response  rate  was 
significantly  different  across  some  of  the  eight  markets. 
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In  the  last  three  to  four  decades ,  considerable  progress  has  been  made  .. 

the  art  of  data  collection  and  in  the  reduction  of  nonsampling  error  in  sur- 

1 

vey  research,'   However,  there  are  still  a  number  of  areas  in  opinion  research 

about  which  either  very  little  is  known  or  the  existing  evidence  is  inconclu- 
sive. This  is  especially  true  for  mail  surveys  where  there  is  very  little 

agreement  about  the  factors  which  affect  response  rate  or  the  magnitude  of 

2 

their  influence  although  considerable  empirical  research  has  been  conducted. 

For  example,  the  interaction,  effects  of  interviewing  method  or  follow-up 
procedures  aad  questionnaire  length  remain  unknown.  We  believe  this  lack  of 
knowledge  is  due  to  at  least  three  causes:  many  variables  influencing  the 
response,  lack  of  a  c     ttensive  theory,  and  limited  resources  allocated  to 
fundamental  research  on  survey  techniques. 


There  are  three  aspects  of  mail  surveys  which  are  of  considerable  practical 
importance  and  yet  are  n  mderstood.  First,  what  are  the  effects  of 
different  follow-up  procedu  •     mail  surveys?  Is  a  post  card  reminder  as 


For  a  comprehensive  bibliography  on  research  on  data  collection  see  D.  1 
Potter  et*  al.  Questionnaires  for  Research:  An  Annotated  Bibliography  on 
Design, r  Construction  'and  Use,  Portland,  Pacific  Northwest  Forest  and  Ra1 
Experiment  Station,'  USDA^  1972 j  and  for  a  comprehensive  theory  of  nonsampling 
error,  See  S.  Sudman  and  N.  Bradburn,  Toward  a  General  Theory  of  Response 
Effects  in  Surveys,  Aidine  Press  (in  pressTT" 
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'Christopher  Scott,  "Research  on  Mail  Surveys",  Journal  of  the  Royal  Stati- 
cal Society,  Vol.  12M-  (2),  1961,  pp.  143-205;  Paul  L.  Erdos,  Professional  Mail 
Surveys,  New  York:   McGraw-Hill  Book  Company,  1970. 
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good  as  a  telephone  reminder?  Is  it  better  to  switch  to  a  telephone  or  per- 
sonal  interview  for  those  respondents  who  do  nor  return  the  mail  questionnaire 

within  a  prespecified  time  period?  While  there  is  a  general  consensus  that 

3 
any  follow-up  or  prodding  generates  additional  response,  there  is  little  agree- 
ment about  the  relative' effects  of  different  follow-up  procedures. 


Second,  what  is  the  effect  of  questionnaire  length  on  the  response  rate?  It 
seems  logical  to  expect  that  response  rate  should  decrease  with  an  increase 
in  questionnaire  length,,  but  most  of  the  evidence  to  date  has  failed  to  support 
this  hypothesis.   In  fact,  the  opposite  result  has  been  found  in  a  number  c 
studies.   Unfortunately,  deductive  reasoning  and  face  validity  are  more  con- 
vincing to  most  researchers  than  isolated  empirical  research,  and  consequently 
response  rate  is  still  believed  to  decrease  with  increased  questionnaire  lengt 

The  third  and  one  of  the  most  important  questions  is  the  determination  of  inter- 
action  effects  among  questionnaire  length,  follow-up  method,  and  various  respond- 
ent characteristics.  For  example,  is  one  particular  follow-up  method  more 
effective  for  a  short:  questionnaire  a  i  another  technique  better  for  a  long 
questionnaire?  Also,  are  some  respondents  more  receptive  to  a  particular  kind 
of  follow-up  method  or  questionnaire  length  than  other  respondents? 

STUDY  DESIGN 

Answers  to  these  questions  about  the  effects  of  questionnaire  length,  follow-up 
method,  respondent  characteristics ,  and  their  interactions  are  critical  to 


3 
J.  R.  Hochstiro  and  D.  A,  Athanasapoulos ,  "Personal  Follow-up  in  a  Mail  Survey: 

Its  Contribution  and  Its  Coat,"  Pub 1 I      'on  Quarterly »  Vol.  3^,  1970,  pp* 

69-61;  and  B.  K.  Eckland,  "Effects  of  Prodding  to  Increase  Mail-Back  Returns," 

JP-'j^^A-g'r.. J^PAAgA ■.^gyG^0:!-0.Sy »  Voi»    ^9»   1965,   pp.    165-169. 
**See  Christopher  Scott,  op*   cit.»  for  a  listing  of  studies. 
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ATST  which  periodically  sends  a  mail  questionnaire  to  its  residence  customer 
panel  consisting  of  30s000  members..  These  customers  are  asked  to  participate 
in  a  survey  by  completing  and  returning  a  four-page  questionnaire,  and  data  are 
used  as  input  for  AT&T's  Market  Research  Information  System  (MRIS).  The  30,000 
residence  customers  in  this  longitudinal  panel  nave  been  chosen  to  be  represen- 
tative of  all  telephone  users  at  the  national  and  state  levels,  and  for  local 
areas  within  some  of  the  larger  states.  To  ensure  representativeness  of  the 
sample  in  the  fixed  panel,  it  is  important  to  maximize  the  questionnaire  response 
rate  to  include  as  many  of  the  panel  members  as  possible  in  the  subsequent 
analysis. 

The  four-page  questionnaire  mailed  to  panel  members  requests  information  on 
telephones  and  their  location  in  the  home,,  telephone  usage,  housing,  mobility, 
demographics ,  and  socioeconomic  characteristics.  Together  with  information 
from  company  billing  records 5  these  data  are  used  to  develop  models  of  tele- 
phone behavior  patterns.   The  survey  procedure  uses  an  alert  post  card  fol- 
lowed by  a  first  mailing  of  the  questionnaire  with  a  cover  letter  asking  the 
respondent  to  cooperate  in  '.    ^rvey   A  reminder  post  card  then  is  sent  fol- 
lowed by  a  second  mailing  of  t       Lonnaire  to  those  who  did  not  fill  out 

5 

and  return  the  first  qu^ 

At  the  start  of      resent       three  years  had  passed  since  the  initial 
customer  data  had  bee,  .  and  it  was  necessary  to  obtain  more  current 

and  up-to-date  information  by  mailing  another  questionnaire  to  present  panel 
members.   At  th<      time,  it  was  decided  to  revise  the  present  four-page 


5This  procedure  is  detailed  in  Paul  L.  Erdos,  on_.  cit_. 


.it- 
questionnaire,  determine  the  effects  of  gathering  additional  attitudinal  in- 
formation, validate  the  present  survey  method,  and  test  alternatives  to  improve 

the  response  rate. 

Since  past  research  evidence  on  th€  effects  of  follow-up  methods  and  question- 
naire length  on  response  rate  was  not  decisive  in  guiding  management  decisions, 
a  study  was  initiated  to  investigate  the  individual  and  joint  effects  of  these 
factors  in  a  number  of  heterogeneous  market  areas.  The  study  consisted  of  a 
2  x  U  x  8  three-factorial  fixed-effect  field  experiment  and  a  subsequent  set 
of  postexperiment  interviews  to  explore  customer  attitudes  toward  the  mail  survey. 

The  first  experimental  factor  was  questionnaire  length*  It  consisted  of  two 
levels:  a  four-page  short  questionnaire  (slightly  revised  from  the  existing 
questionnaire)  and  a  six-page  long  questionnaire  which  included  additional  atti- 
tudinal and  perceptual  questions  about  the  telephone  as  a  product  and  as  a  ser- 
vice. The  short  questionnaire  consisted  of  2     nts  and  the  long  questionnaire 
consisted  of  the  same     terns  plus  26      -dinal  items.  The  questionnaire 
was  pre    id  en  a  sample  of  33  respondents  in  selected  areas  for  wording ^  se- 
quence and  format. 


The  second  factor  war  folio       hods.   Four  different  follow-up  methods 
were  selected  as  feasible  alternatives  after  considerable  exploration  of  the 
cost  and  convenience  of  various  follow-up  strategies.   The  four  interviewing 
methods  selected  used  a  uniform  post  card  alert  and  first  questionnaire  mail- 
ing.  The  follow-up  methods  varied  among  experimental  treatments  and  consisted 
of  the  present  post  card  and  three  other  methods: 


1.  Present  follow-up  method  -  post  card  reminder  and  a  second  mailing 

of  the  questionnaire 

2.  Second  fa  ad   -  telephone  reminder  and  a  second  mailing 

of  the  questionnaire 

3.  Third  follow-up  m<        -  letter  alert  followed  by  a  telephone 

erview 

4.  Fot  ■  thod   -  telephone  interview  without  aler»t 


The  third  factor  was  the  market  heterogeneity.  The  national  panel  in  the  MRIS 
system  is  based  on  a  multistage  stratified  sample  in  which  the  Bell  System  is 
geographical        ed  into  one  hundred  regions,  and  a  sample  of  300  residence 

customers  is  chosen  to  represent  each  region.  Based  on  a  hybrid  multivariate 

6 
cluster  analyse    r  the  100  kets  in  terms  of  socio-economic, 

demographic  and  telephone  usage  characteristics,  the  following  eight  geographi- 
cal areas  were  chosen  as     esentative  market  regions:  Southeast  Massachusetts, 
San  Jose,  Arizona ,  tern  Wisconsin,  Suburban  Chicago s  Fort  Worth 

and  rep:  seated  eight  fixed  levels  of  the  third  factor 

.  in  ental  desij 

Within-  each  test  maj  were  mailed  to  a  representative  random 
probabi.j. '  Southea    [assachusetts ,  296  question- 
nail  .''  disconnection 
of  eason,  and  the  results  were 
correspond  .  test  region,  the  sample  was 
equally  nditions  (two  questionnaire 
lengths  and  the  four  follow-up  lures)  with  the  result  that  each  of  the  64 
cells  iv,                   o- fact .  '                     a  sample  of  33  customers. 

■:.  Roscoe  and  J.  N„  Sheth,  "Applications  of  a  Hybrid  Cluster  Analys  ■ 
in  Industrial       iident      'kets  ,rr  unpublished  working  paper,  January  1974. 
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In  addition  to  these  three  experimental  factors,  information  on  respondent 
characteristics  was  obtaj     -  using  the  demographic,,  socioeconomic,  tele- 
phone usage,  and  attitudii  tion  from  the.  questionnaire  itself.  These 
data  were  used  to  estiraat      respondent  attributes  which  could  Interact  with 
questionnaire  length  and  follow-up  m      to  influence  reaction  to  question- 
naire let;'     id  follow-up  methods. 

The  study  began  with  the  2,144  members  of  the  sample  receiving  an  alert  post- 
card on  or  about  Monday,  September-  18,  1972  and  the  questionnaire  on  Wednesday, 
September  20*  The  post  card  and  telephone  reminder  groups  received  a  second 
questionnaire  en  Friday,  September. 29 ► 

The  telephone  reminders  and  follow-up  telephone  interviews  were  conducted  from 
September  29  through  October  2  by  the  field  staff  of  a  national  commercial  r^ 
search  company  in  each  test  region.  The  mailing  dates  including  those  for  the 
follow-up  procedu      e  designed  in  such  a  way  as  to  compensate  for  the  re- 
gional variations  in  postal  delays.  A  total  of  four  attempts  were  made  to  c 
tact  each     ndent  for  the  telepho;  5  reminders  and  telephone  interviews.  In 
addition,         viewer  was  ail--  ;  to  any  responsible  ad  ait  in  the 

household  if  the  p«  tele-     ■ tiling  account  was  not  avail- 

able. Fin  .  the  day  and  evening  which 

the  local  supervisor  co:  .  priate.  In  short,  every  possible 

fort  was  made  to  minimize  the  6  -nces  in  situational  factors  (time,  person 

and  opportunity  to  reach.   tween  the  telephone  and  mail  interviews. 

The  mail  questionnaires  were  accompanied  by  cover  letters  on  the  letterh 

the  local  Bell  Telephone  Company.  the  concent  of  the  alert  post  card, 


the  cover  letter  and  the  reminders  was  .identical  across  all  the  eight  test 
market  regions.  All  mai   gs  were  posted  by   the  survey  research  manager 
each  local  Bell  Telephone  Company  and  the  return  mail  was  delivered  to  that 
company.   First  class  max]  was  used  for  both  the  delivery  and  the  return  of 
the   questionnaire  be cans  and  special  delivery  mails  are  not  only 

costly  but  were  Lor  resi  a  source  of  irritation  to  t 

respondents . 

Post- experiment  interviews  were  conducted  with  a  subsample  of  respondents  in- 
cluding those  who  did  not  reti     te  questionnaire  by  the  end  of  the  experimen 
These  interviews  were  conducted  to 'determine  the  respondent's  feelings  toward 
the  survey  in  general,  and  their  reactions  to  the  time  intervals  in  the  mailing 
and  follow-up  proce<         srify  the  demographic  information  provided  in 
the  survey j  and  to  che    le  understanding  of     fie  questions-  Interview- 
ing began  en  October  by  November  6.  A  total  of  393  respond- 
ents were  successfully  interviewed  and  were  approximately  equally  divided  among 

three  experimental  factors,   One-third  of  Lews  were  conducted 

person  and  the  others  b}      hone. 

An  analysis  a    dance  war  .e   basic  experiment,,  and 

the  teractions  w  i 

tasted  for  significance.  In  addition,  sp<      quests     i  the  test  q. 
naire  were  crosstabulated  by  que  i     lire  length,  follow-up  method,  and  mark- 
From  the  post-experimei  ge  and  attitude  information     tabulal 

by  whether  or  net  the  customer  returned  the  earlier  experimental  questionna 
questionnaire  length,  and  follow-up  method.  The  post-experiment  survey 
responden'  .  -ondents  tc     sompared  in  terms  of  other  experimental 

factors. 
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RESULTS  MD  DISCUSSION 

The  results  of  the  study  c.re  surra  n  Table  i.  The  overall  response 
rate  was  67.3  percent.  However 5  theri  are  significant  differences  across 
various  experimental  treatments  of  tne  three  factors. 

First,  the  four  fo     ip  methods  produ      nsiderabiy  different  response 
rates  across  the  mar  .  the  questionnaire.   The  tele- 

phone reminder  was  the  best  follow-up  with  an  average  response  rate  of  76.4 
percent,  followed  by  t        ard  reminder   tl  an  average  of  69,6  percent t 
the  telephone  Interview  with  an  alert  an  average  of  65.9  percent,  and  finally 
the  telephone  interview  without  an  alert  with  an  average  of  57.0  percent  respc 

rate.   Based  on  the  Newman-Keuls  test  on  the  pairwise  differences  within  an  e; 

7 
peridental  factor,  we  found  that  most  of  the  differences  across  follow-up 

methods  can  be  attributed  to  these  two  conditions:  telephone  interview      .,_ 

alert  is  significantly  woi     p  <  ,01)  than  the  other  three  follow-up  methods, 

and  the  telephon  Ls  significantly  better  (p  <C  -05)  than  the  oth2r 

r-ee  follow-up  procedures  in  generating  the  response  rate* 

Second t   there  are  no  ences  In  the  response  rate  between  J 

short  and  the  1c  rrary  to  t    cpectations.  Our  resu] 

thus  add  to  the  gr  length  of  the  questionnaire,  with"' 

r-^d:     e  limits,,  ffects  on  the  response  rate.  In 

fact,  we  fin  tiough  nonsignificant  improvement  in  the  response 

with  the  long  pe» 


See  B.  J.  Winer,  Statistical  peridental  Design,  Seccr. 

.  York,  McGraw-Hill  Book  Company,  1971,  pp.  191-196. 
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Third,  as  expected,  the  eight  markets  performed  differently  to  the  experi- 
ments.! conditions.  The  best  response  rate  across  various  experimental  condi- 
tions came  from  the  Fort  Worth  area*  and  the  worst  response  rate  came  from  the 
Alabama  area.   Once  again,,  by  using  the  Newman-Keuls  test,  we  found  that  most 
of  the  differences  across  the  eij        :s  came  from  these  conditions:  Alabama 
had  a  significantly  lower  response  rate  than  all  other  markets  (p  ^  .05)  and 
Fort  Worth  had  a  significantly  higher  response  rate  than  all  other  markets 
(p  <  =05). 

In  order  to  quanti      significant  differences  across  the  three  experimental 
factors  and  to  isolate  the  main  and  the  interaction  effects,  an  analysis  of 
variance  was  perforate        data  utilizing  the  Tukey's  test  for  nonadditivity. 
The  resul        inrniarized  in  Table  2.  As  expected  *  both  the  factors  of  follow- 
up  methods  and  market      ageneity  had  significant  main  effects  and  the  ques- 
tionnaire length  had  no  signifies     in  effects.  We  also  found  significant 
interaction  effects  between     :ollow-up  methods  and  the  market  heterogeneity 
actors  and  tc  rent  between  questionnaire  length  and  marke 

i  ] 

The  imental  survey  i  d  to  assess  attitudinal  differenc 

among  re*  nt  experimental  conditions.  We  shouJ 

expect,  no  significant  di     ices  a     experimental  groups  in  their  gene* 

titudea  toward  .      partic  >th  the  responders  and  nonrespond- 

are     ,ded  in  er>  -      mental  group.  This  lack  of  difference  can  suggest 

that  the  e&perimentaJ  homogeneous  in  their  feelings  toward  sui  ■ 

8 

See  J.  B.  .Miner     cit»»  pp.  39i*-397» 


participation  which  can  provide  additional  credence  to  the  analysis  of  variance 

result.. 

Table  3  summarises  th  snts  toward  survey  participation 

across  different  expe^  have  not  broken  down  further  by 

mark     le  to  extreme  zes  for  many  categories  of  responses 

coded  from  open  st-experiment  survey.  As  can 

be  seen  from  Table  !       srcenta      positive,  negative  and  neutral  corn- 
merits  are  remarkably  similar  across  the  experimental  conditions  when  both 
responders  and  nonre spenders  ere  grouped  together  in  each  experimental  condi- 
tion. On  the  average,  81  percent  of  the  respondents  had  positive  feelings  toward 

•vey.  While  the  degree  or  manner  of  positive  feelings  varied  somewhat 
across  experimental  groups  t   generally ,  there  is  a  remarkable  degree  of  similar- 
ity in  the  da     A  total  of  15  percent  of  the  respondents  had  negative  feelings 
tow;        -      and  the  .  .      f  7  percent  were  either  neutral  or  had  no 

iosaogeneity  of  experimental  groups  with  respect  to  their  general 
feelings  toward  survey  participation  gives  stronger  credence  to     results  dis- 

earlier. 

In  order  to  obtain  insights  into  the  reactions  toward  the  specific  experimental 
conditions  (.  th  and  follow-up  procedures),  the  respondents  were 

directly  a;  about  the  spec:-         .mental  condition  to  wh: 

they  were  subjected.  Te1  :e  feelings  toward  the  length  of  the 

stionnaire  fo  s   of  the  experimental  condition.  It 

should  he  nc     iat  only  those  who  cooperated  in  the  survey  (returned  the  ques- 
tionnaire) are  included  in  this  analysis.  As  can  be  seen  from  the  table,  there 
are  no  real  differences  be     the  two  groups  of  short  and  long  questionnaires 
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although  there  is  a  si     /  more  Incidence  of  positive  comments  in  the  longer 
questionnaire  group.  More  Interestingly,  the  group  with  longer  questionnaire 
considered  it  to  be  "comprehensive /well  designed"  significantly  more  than  the 
group  with  the  shorter  questionnaire.  However »  this  is  somewhat  offset  by  a 
higher  percentage  in  1  questionnaire  group  with  respect  to  "easy  to 

fill  out/not  too  long      "questions  were  to  help  the  company  improve  service" 
categories  of  positive  responses.  With  respect  to  the  negative  comments,  it  is 
surprising  to  find  that  considerably  more  respondents  in  the  shorter  question- 
naire group  felt  that  some  of  the  questions  were  too  personal.  As  would  be  ex- 
pected, slightly  more  number  of  respondents  in  the  longer  questionnaire  group 
felt  that  some  of  the  questions  were  complicated. 

Table  5  summarizes  the  pc  lit  comments  specifically  related  to  the 
follow-up  methods.  Due  to  the  small  sample  size,  we  have  combined  respondents 
for  both  telephone  interview  follow-up  procedures  into  a  single  category.  The 
are  .  differences  across  the  three  groups  between 
ir.  s  and  the  specific  types  of  positive  com- 
ments. Signii  spondents  in  the  telephone  interview 
gro1  >ne  reminder  and  the  post  card 

dnder  procedures  (8?  r  vs.  51  and  u?  percent,  respectively).  This  is 
somewhat  su  ,  of  the  fact  that  the  response  rate  was  lower  in  the 
tele          .-lew  fc  ^r ,  the  bulk  of  the  positive  com- 
ments is  more  in  th  \»  ^   ingratiating  the  telephone  in- 
terviewer than  .:  itage  e       >llow-up  method.  Thus,  41  perc- 
stared  that  they  ■-'  '-aw  an<i  30  percent  stated  that 
they  were  glad  to  hel  company.   In  contrast,  only  10  percent  felt  that  phone 
call  nade  the  s       earer  and  27  percent  favored  telephone  interview  over  I 
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mail  questionnaire.  Examining  the  positive  comments  in  the  telephone  and 
post  card  reminder  groups,  however,  we  find  that  most  of  the  positive  commei 
are  directly  anchored  to  the  method  of  follow-up  procedures.  Many  respondents 
considered  the  specific  follow-up  procedure  a  good  idea  or  a  good  reminder. 

Turning  toward  the  negat i  Lnd  that  some  of  the  cooperative  re- 

.dents  in  the  telephone  int«  :inft  like  telephone  interviews 

or  resented  the  telephone  survey      they  had  an  unlisted  number.   Surprisingly, 
some  of  the  respondents  in  the  telephone  reminder  procedure  found  it  to  be 
'•annoying  or  pushing  me," 

Finally,  there  is  a  large  percentage  of  respondents  who  claimed  that  they 

seive  the  post  card  reminder  or  didn't  remember  receiving  it  (39  per- 
cent). Similarly,  some  c    a  respondents  in  the  telephone  reminder  group  also 
claimed  not  receiv     ie  reminder  or  not  remembering  it.  We  believe  that 

se  arc  excuses  for  no;     Ly  participating  in  the  survey.  Furthermore,  It 
is  easier  for  the  respon i      claim  nonreceipt  of  the  reminder  if  it  is 
through  the  mail  than  ugh  the  telephone.  In  fact,  the  superiority 

hone  n  n  the  study  may  be  solely  due  to  this  factor. 

Our  analye  .duced  no  significant  differences 

it  the  re-     nts  in  had  significantly  higher 

income  than  the  1970  Census       Data  for  the  area.   Due  to  a  very  high  devel- 
opment of  multiple  te  ipper  income  households  in  the  Fort 
Worth  area,  the  random  probability  sample  had  generated  more  numbers  ot  upper 
income  respondents.  The  lack  of  systematic  differences  in  the  feelings  toward 
survey  participation  or  toward  specific  experimental  conditions  across  the  eight 
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markets  is  disappointing  as  we  strongly  believe  that  we  need  to  consider  the 
concept  of  market  segmentation  in  survey  research t  some  questioning  procedures 
are  better  suited  foi  .    segment  of  the  total  respondents  and  others  are 
better  suited,  for  other  segments.  No  single  technique  of  data  collection  can 
fully  satisfy  all  segments  of  the  population. 

SUMMARY  AND  CONCLUSIONS 

This  study  was  designed  to  test  and  evaluate  two  specific  hypotheses  for  manage- 
ment.  First9  could  an  instrument  of  26  attitude  items  be  added  to  the  present 
questionnaire  without  deleterious  effects  on  response  rate?  Second,  is  there 
3  practical  alternative  to  the  present  follow-up  procedure  that  would  increase 
overall  response  rate.?  The  results  clearly  indicated  that  the  attitude  questions 
can  be  added  to  the  questionnaire  without  any  measurable  effect  in  response  rate 
and  that  this  revised  larger  questionnaire  would  not  influence  the  selection  of 
;  follow-up  procedure.  t  _ephone  reminder  with  the  second  mail  question- 

naire is  clearly  the  best  overall  strategy  to  replace  the  present  post  card  re- 
The  telephone  reminder  was  consistently  higher  over  all  markets  and 

able  impassion  on  the  survey  participants. 
The  emphasis  here  was      d  on   increased  response  rate  which  would  result  in 
improved  panel  validity  lerefore ,  no  direct  cost  compari- 

sons were  made.   However,  since  the  ng  and  collection  are  done 

locally,  the  telephone  costs  compare  favorably  with  the  cost  to  print,  address 
and  mail  the  reminder  post  car: 

Since  the  telephone  reminder  and  the  post  card  reminder  are  similar  in  nature 
and  diffe:      in  form,  it  is  possible  to  suggest  that  if  specific  markets 
find  that      esults  with  the  telephone  reminder  do  not  achieve  a  better  retv 
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than  •  previous  experience,  they  should  continue  to  use  the  post  card 
reminder.  This  tailoring  of  the  survey  procedure  to  the  specific  markets 
suggests  that,  the  besl  might  be  different  for  various 

market  segme  t  ar,  the  results  suggest  that  high  socioeconomic 

customer  group?  rest  respond  best  to  the  alert  and  tele- 

phone interview.  ure  should  not  be  used  in  the  Southern 

areas  where  the  best  r       was  obtained  by  a  telephone  reminder.  The 
telephone  intervie^  t  recommended  as  of  follow-up  procedures  in  and 

around  the  major  metr      an  areas  where  the  telephone  reminder  was  preferred 
in  the  city  and  the  post  card  reminder  in  the  suburban  areas.  Additional  re- 
search, however,  is  necessary  to  confirm  and  extend  these  findings, 
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TABLE  2 

ANALYSIS  OF  VARIANCE  OF  QUESTIONNAIRE  RETURNS* 


Source  of  Variation 
Questionnaire  Length 
Follow-up  Methods 

Markets 

Questionnaire  Length  X 
Follow-up 

Questionnaire  Length  X 
Market 

Follow-up  X  Market 

Questionnaire  Length  X 
Follow-up  X  Market  (Error) 


Sum  of    Degrees  of 

Squc-      Freedom    F  Ratio   Significance 


0.26 
342.53 
350,93 

7aSl 

103,70 
311.87 

14-1.58 


3 
7 

3 

7 

21 

21 


0.04 

16.94 

7.44 

0.37 

2.20 
2.20 


N.S. 
.001 
.001 

N.S. 

.10 
.05 


*The  Anova  procedure  is  based  on  B.  J.   Winer »  St  at  is t ical  Pr inc iple s _ in 
Experimental  Design,  2nd  Edition,  New  York;  M7-Gra^ETiT~Book  Company,  1971, 
pp.  321-327  and  394-397.  The  use  of  the  three  factor  interaction  for  the 
estimate  of  the  error  variance  was  based  on  Tukey's  test  for  non-additivity 
which  *     ■  cted  as  not  significant. 
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TAELE  3 


ATTITUDES  TOWARD  SURVEY  PARTICIPATION* 


Respondents  Making  a 
££Sitrve_Coifflnent 

Happy  to   do  it /Glad  to 
help /An  honor 

Didn't  mind/Was  OK 

Gave  chance  to  express 
feelings /Helping 
Improve  service /To  irry 
benefit 

Like  Phone  Co/Happy  they 
were  interested/Good  way 
to  find  out  about  service 

Questions  were  easy 


Quest  i.- 

gth 


Follow-up  Method 


Telephone  Telephone 
Telephone  Post  Card  Interview  Interview 
h2BL^^^hS£l^.^3^^^       Reminder   No  Alert   with  Alert 


92)  (n=194)   (n*97)     (n=97)     (n=96)     (n=96) 


81% 


11 

15 
3 


80% 


15 

12 
2 


82% 


17 

20 
0 


81% 


10 

1 


80% 


11 

17 
4 


79% 


27 

18 

25 

20 

26 

20 

43 

43 

35 

47 

45 

46 

12 

9 

5 


Making  a 
Neutral  Comment 


Respondents  Making 
Negative  Comment 


Don't  like  Phone  Co/ 
Unh  a  pp  j.      s  erv  i  ce  / 
Survey  not  worthwhi 

Questions  too  personal/ 
Too  long/Confua 

Didn't  have  time 
An  imposition 


15 


16 


13 


10 


16 


14 


18 


*Eaaed  on  all  386  respondents  to  the  post-experiment  survey  including  those  not  returning 

the  test   questionnaire. 
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TABLE  H 


ATTITUDES  TOWARD  THE  QUESTIONNAIRE'' 


Respondents 

Pos  i  t  i  vg  Coj^g n t 

Was  good/nice/fine 

Didn't  mind  it 

Comprehensive /We 11  des igned 

Gave  opportunity  t<     ess 
fee 

Easy  to  fill  out/3 

Questions  were  to  help  th< 
my  improve,  service 


LO: 

1 

Short 

Quest  ionnaire 
(n=159) 

Ques- 

fc ionnaire 
1-161) 

67% 

26 

19 

15 

16 

28 

17 

1.1 

9 

10 

16 

11 


pondents  Makii      tral 
sent 


its  Making  a 
Ne  gat  i  ve  Comment 

ticns  too  pe 

compl.i 

Some  questions  iiv  |      ite/ 

sh 

her  neg        lents 


30 

12 
10 


30 

14 

8 

9 
6 


*Based  o    •  320  respondents  to  tl     t-experiment  survey  who  had  completed 
the  test  questionnaire. 
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TABLE  5 
ATTITUDES  ABOUT    i  DW-UP  METHODS 


Respondents  Making  a 
'Positive   Comment 

Good  idea/Good  reminder 

Made  it  seem  impo:  tanl  to 
respond 

Was  fine/Didn't  mind 

Showed  they  care 

Rather  telephone  interview 
than  questionnaire 

Phone  call  made'  survey  clearer 

Caller  was  cour  a   ,  pleasant 

id   to  help 


Telephone 

Post  Card 

Telephone 

Remind* 

Reminder 

Interviews* 

(n=97) 

Xn=97) 

Cn=41) 

51% 

47% 

87% 

22 

NA 

8 

9 

0 

0 

30 

41 

6 

0 

0 

NA 

NA 

27 

m 

NA 

10 

23 

NA 

17 

0 

0 

30 

pendents  Making  a 

Annoy e  d/Push  ing  me 

Don't  like  telephone  interv 

■■    ■ 
ent  the  call 


11 

1.1 
NA 


7 

5 

NA 

NA 


18 

0 

15 


Don ' t  I' 


-f..-L 


39 


art  group  and  no  alert  grou]     bined.   Includes  only  respondents  given 
telephone  interview. 


